In this work, we improve on the upper and lower bounds for the regret of online learning with strongly observable undirected feedback graphs. The best known upper bound for this problem is \mathcal{O}\bigl(\sqrt{\alpha T\ln K}\bigr), where K is the number of actions, \alpha is the independence number of the graph, and T is the time horizon. The \sqrt{\ln K} factor is known to be necessary when \alpha 1 (the experts case). On the other hand, when \alpha K (the bandits case), the minimax rate is known to be \Theta\bigl(\sqrt{KT}\bigr), and a lower bound \Omega\bigl(\sqrt{\alpha T}\bigr) is known to hold for any \alpha . Our improved upper bound \mathcal{O}\bigl(\sqrt{\alpha T(1 \ln(K/\alpha))}\bigr) holds for any \alpha and matches the lower bounds for bandits and experts, while interpolating intermediate cases.

Update after rebuttal I thank the authors for a comprehensive rebuttal and extra experiments. It has addressed most of my concerns, and I have updated my score. The authors should make sure to properly tone down the claims about improved training for LDA (vs. It seems to me that we do not really understand very well what is happening in these models at this stage; this perplexity experiment is just scratching the surface (and should be presented as such). I am also a bit puzzled by the use of alpha 1.001 (vs.

Additional Feedback: To be honest, I find the term "double overparametrization" a bit strange. I would still call it simply "overparametrization". Perhaps, the authors could think about this point and potentially adjust. I would suggest that the authors briefly discuss the following point which is sometimes overlooked when discussing implicit bias of gradient descent in the context of low rank matrix recovery. When additional restricting to positive semidefinite matrices it turns out that the original low rank matrix is often the UNIQUE solution to the linear equation y A(X) that is positive semidefinite, see the paper "Implicit regularization and solution uniqueness in over-parameterized matrix sensing" by Geyer et al., arxiv:806.02046,

Weaknesses: W1: The study seems to focus too much on algorithms that are based on safety tests. I understand that the analysis is not compatible, but maybe that would be worth it to include studies on how easy it is to trick those algorithms too. More generally (even for IS algorithms), it was a bit odd to me that the study does not consider attacks on the way pi_e is chosen. W2: It's unclear to me whether the trajectory must still have been performed in the real environment, or it can be completely be made up (but then its value has to be within the range [0,1]). Also, with model based methods (for both environment and policy models), it might be possible to single out the few trajectories that are inconsistent with the other trajectories.

Given \alpha,\epsilon, we study the time complexity required to improperly learn a halfspace with misclassification error rate of at most (1 \alpha)\,L *_\gamma \epsilon, where L *_\gamma is the optimal \gamma -margin error rate. For \alpha 1/\gamma, polynomial time and sample complexity is achievable using the hinge-loss. An immediate question, which this paper tackles, is what is achievable if \alpha \in (0,1/\gamma) . We derive positive results interpolating between the polynomial time for \alpha 1/\gamma and the exponential time for \alpha 0 . In particular, we show that there are cases in which \alpha o(1/\gamma) but the problem is still solvable in polynomial time.

Continual learning (CL) is widely regarded as crucial challenge for lifelong AI. However, existing CL benchmarks, e.g. Permuted-MNIST and Split-CIFAR, make use of artificial temporal variation and do not align with or generalize to the real-world. In this paper, we introduce CLEAR, the first continual image classification benchmark dataset with a natural temporal evolution of visual concepts in the real world that spans a decade (2004-2014). We build CLEAR from existing large-scale image collections (YFCC100M) through a novel and scalable low-cost approach to visio-linguistic dataset curation. Our pipeline makes use of pretrained vision-language models (e.g. CLIP) to interactively build labeled datasets, which are further validated with crowd-sourcing to remove errors and even inappropriate images (hidden in original YFCC100M). The major strength of CLEAR over prior CL benchmarks is the smooth temporal evolution of visual concepts with real-world imagery, including both high-quality labeled data along with abundant unlabeled samples per time period for continual semi-supervised learning. We find that a simple unsupervised pre-training step can already boost state-of-the-art CL algorithms that only utilize fully-supervised data. Our analysis also reveals that mainstream CL evaluation protocols that train and test on iid data artificially inflate performance of CL system. To address this, we propose novel "streaming" protocols for CL that always test on the (near) future. Interestingly, streaming protocols (a) can simplify dataset curation since today's testset can be repurposed for tomorrow's trainset and (b) can produce more generalizable models with more accurate estimates of performance since all labeled data from each time-period is used for both training and testing (unlike classic iid train-test splits).

This post shows how to use the R packages for estimating an exclusive lasso and a group lasso. These lasso variants have a given grouping order in common but differ in how this grouping constraint is functioning when a variable selection is performed. Lasso, Group Lasso, and Exclusive Lasso While LASSO (least absolute shrinkage and selection operator) has many variants and extensions, our focus is on two lasso models: Group Lasso and Exclusive Lasso. Before we dive into the specifics, let's go over the similarities and differences of these two lasso variants from the following figure. In the above figure, 15 variables are categorized into 5 groups.

Given $\alpha,\epsilon$, we study the time complexity required to improperly learn a halfspace with misclassification error rate of at most $(1 \alpha)\,L *_\gamma \epsilon$, where $L *_\gamma$ is the optimal $\gamma$-margin error rate. For $\alpha 1/\gamma$, polynomial time and sample complexity is achievable using the hinge-loss. For $\alpha 0$, \cite{ShalevShSr11} showed that $\poly(1/\gamma)$ time is impossible, while learning is possible in time $\exp(\tilde{O}(1/\gamma))$. An immediate question, which this paper tackles, is what is achievable if $\alpha \in (0,1/\gamma)$. We derive positive results interpolating between the polynomial time for $\alpha 1/\gamma$ and the exponential time for $\alpha 0$.

Our third regional GPU Technology Conference in as many weeks reached another packed house today, as NVIDIA co-founder and CEO Jen-Hsun Huang unveiled technology that will accelerate the deep learning revolution. "GPU computing is at the beginning of something very, very important, a brand new revolution, what people call the AI revolution, the beginning of the fourth industrial revolution," Huang told a crowd of 1,600 scientists, engineers, entrepreneurs and press, gathered at Amsterdam's gleaming waterfront music hall. "However you describe it, we think something really big is around the corner." In the latest stop in a tour that will bring GTC to eight cities around the world, Huang unveiled Xavier, our next-generation system-on-chip for powering self-driving cars; announced an agreement with TomTom, the Dutch mapping and navigation group, to use AI to create a cloud-to-car mapping system for self-driving cars; detailed our DriveWorks Alpha 1 release, and highlighted the work we're doing with some of Europe's most innovative startups and research labs. In the previous two weeks, Haung spoke at regional GTCs in Beijing and Taiwan that each drew crowds of more than 2,000.

Lines for the iPhone 7 wrapped around the block outside of Apple stores worldwide leading up to today's highly-anticipated launch, and as always, many hopefuls camped out for days to secure a spot. But, with the help of 100 tiny robots, some customers managed to be among the first to get their hands on the device without ever having to leave their homes. New Zealand firm Spark enlisted a fleet of Alpha 1 robots to stand in line in place of their human counterparts – and they can dance, do kung Fu, and live-stream their way through the queue. The'robot army' is made up of devices from Chinese company UBTECH, and each bot is paired with a Spark customer who can control their actions using a smartphone app. This means the robots can be made to do push-ups, Kung Fu, yoga, mimic Olympic sports, and dance.